Image processing apparatus, method, and program

ABSTRACT

An image processing apparatus, including a region setting unit for setting regions on a processing target image in which each pixel of the image is included according to a level of each pixel, a low frequency image generation unit for generating a low frequency image of the processing target image, a gain calculation unit for calculating a gain for each pixel of the processing target image such that the lower the level the greater the gain, wherein the unit calculates the gain such that pixels in each of the regions in which each pixel of the processing target image is included have substantially the same gain based on a region setting result and a level of each pixel of the low frequency image, and a processing unit for generating a processed image by performing dynamic range compression on the processing target image based on the gain.

BACKGROUND OF THE INVENTION

1. Field of the Invention

The present invention relates to an image processing method andapparatus for compressing the dynamic range of an image by changing thegain of the image. The invention also relates to a computer readablerecording medium on which is recorded a program for causing a computerto perform the image processing method.

2. Description of the Related Art

Reproduction of images after performing appropriate image processing iscommon practice in various fields. For example, a dynamic rangecompression method is proposed. The method generates a low frequencyimage of an image and reduces the contrast of a high luminance region, alow luminance region, or the entire image such that the differencebetween the maximum luminance value and minimum luminance value, i.e.,dynamic range is reduced using the low frequency image. Morespecifically, a low frequency image of an image is generated, then again is calculated from the low frequency image such that the darker(i.e., the lower the luminance value) the low frequency image the higherthe gain, and the calculated gain is added to each pixel or each pixelis multiplied by the gain.

The dynamic range compression allows for making a bright region of animage darker and a dark region of the image brighter. Thus, for example,a high contrast image of a person taken against light can be turned intoa high quality image by correcting halation in the background and darkcollapsed person's face.

In the dynamic range compression method described above, however,artifacts, such as overshooting, undershooting, and the like occuradjacent to an edge on an image, which cause a problem of image qualitydegradation. Consequently, methods for preventing such artifacts areproposed as described, for example, in Japanese Unexamined PatentPublication No. 10(1998)-075364 (Patent Document 1) and JapaneseUnexamined Patent Publication No. 5(1993)-300376 (Patent Document 2).The method described in Patent Document 1 compresses the dynamic rangeof an image by generating a plurality of band-limited image signals ofdifferent frequency bands from an image signal representing the image,generating a cumulative signal by integrating the band-limited imagesignals in which some of them are reduced before integrated, convertinga differential signal obtained by subtracting the cumulative signal fromthe image signal of the original image by a predetermined function, andadding the signal obtained by the conversion to the original image. Themethod described in Patent Document 2 is a method that calculates acorrection value for dynamic range compression by making comparison inmagnitude of pixel value between an image and a low frequency imagethereof.

The method described in Patent Document 1, however, is not able tocompletely eliminate overshooting or undershooting adjacent to an edgeincluded in an image, since the dynamic range compression is performedbased on frequency separation of the image. The method described inPatent Document 2 may cause a processed image to be discontinuous at aposition where the correction value is changed, since the correctionvalue is calculated based on the magnitude of pixel value between animage and a low frequency image thereof.

The present invention has been developed in view of the circumstancesdescribed above, and it is an object of the present invention to improveimage quality of a dynamic range compressed image.

SUMMARY OF THE INVENTION

A first image processing apparatus of the present invention is anapparatus, including:

a region setting unit for setting one or more regions on a processingtarget image in which each pixel of the image is included according to alevel of each pixel;

a low frequency image generation unit for generating a low frequencyimage of the processing target image;

a gain calculation unit for calculating a gain for each pixel of theprocessing target image such that the lower the level the greater thegain, wherein the unit calculates the gain such that pixels in each ofthe one or more regions in which each pixel of the processing targetimage is included have substantially the same gain based on a regionsetting result and a level of each pixel of the low frequency image; and

a processing unit for generating a processed image by performing dynamicrange compression on the processing target image based on the gain.

In the image processing apparatus described above, the gain calculationunit may be a unit that calculates a provisional gain for each pixel ofthe low frequency image such that the lower the level the greater thegain and, based on the region setting result, calculates arepresentative value of the provisional gains of pixels in each of theone or more regions in which each pixel of the processing target imageis included as the gain for each pixel of the processing target image.

As for the “representative value of the provisional gains”, the averagevalue, weighted average value, or intermediate value of the provisionalgains, or the like, may be used.

Still further, in the image processing apparatus described above, thegain calculation unit may be a unit that calculates, based on the regionsetting result, a representative value of pixels in each of the one ormore regions in which each pixel of the low frequency image is included,and calculates the gain for each pixel of the processing target imagesuch that the lower the level of the representative value the greaterthe gain.

As for the “representative value of pixels”, the average value, weightedaverage value, or intermediate value of the pixels, or the like, may beused.

A second image processing apparatus of the present invention is anapparatus, including:

a low frequency image generation unit for generating a plurality of lowfrequency images of different frequency bands from a processing targetimage;

a region setting unit for setting one or more regions on at least one ofthose of the plurality of low frequency images up to a predeterminedfrequency band, which is higher than a first frequency band, in whicheach pixel of the at least one low frequency image is included accordingto a level of each pixel of the at least one low frequency image;

a gain calculation unit for calculating a first gain for each pixel of afirst low frequency image of the first frequency band such that thelower the level the greater the gain, calculating, based on a regionsetting result, a representative value of the first gains of pixels ineach of the one or more regions in which each pixel of a second lowfrequency image of a second frequency band, which is next higher thanthe first frequency band, is included as a second gain of each pixel ofthe second low frequency image, and outputting a second gain calculatedby repeating the calculation of a new second gain using a low frequencyimage of the next higher frequency band than that of the second lowfrequency image as a new second low frequency image and the second gainas a new first gain to the predetermined frequency band as a final gainof each pixel of the processing target image; and

a processing unit for generating a processed image by performing dynamicrange compression on the processing target image based on the gain.

A photographing apparatus of the present invention is an apparatus,including:

an imaging unit for obtaining a processing target image by photographinga subject; and

the first or second image processing apparatus of the present invention.

A first image processing method of the present invention is a methodincluding the steps of:

setting one or more regions on a processing target image in which eachpixel of the image is included according to a level of each pixel;

generating a low frequency image of the processing target image;

when calculating a gain for each pixel of the processing target imagesuch that the lower the level the greater the gain, calculating the gainsuch that pixels in each of the one or more regions in which each pixelof the processing target image is included have substantially the samegain based on a region setting result and a level of each pixel of thelow frequency image; and

generating a processed image by performing dynamic range compression onthe processing target image based on the gain.

A second image processing method of the present invention is a methodincluding the steps of:

generating a plurality of low frequency images of different frequencybands from a processing target image;

setting one or more regions on at least one of those of the plurality oflow frequency images up to a predetermined frequency band, which ishigher than a first frequency band, in which each pixel of the at leastone low frequency image is included according to a level of each pixelof the at least one low frequency image;

calculating a first gain for each pixel of a first low frequency imageof the first frequency band such that the lower the level the greaterthe gain;

calculating, based on a region setting result, a representative value ofthe first gains of pixels in each of the one or more regions in whicheach pixel of a second low frequency image of a second frequency band,which is next higher than the first frequency band, is included as asecond gain of each pixel of the second low frequency image; and

outputting a second gain calculated by repeating the calculation of anew second gain using a low frequency image of the next higher frequencyband than that of the second low frequency image as a new second lowfrequency image and the second gain as a new first gain to thepredetermined frequency band as a final gain of each pixel of theprocessing target image; and

generating a processed image by performing dynamic range compression onthe processing target image based on the gain.

Each of the first and second image processing methods of the presentinvention may be provided as a program recorded on a computer readablerecording medium for causing a computer to perform the method.

According to the first image processing apparatus and method of thepresent invention, one or more regions is set on a processing targetimage in which each pixel of the image is included according to a levelof each pixel, and further a low frequency image of the processingtarget image is generated. Then, based on a region setting result and alevel of each pixel of the low frequency image, the gain is calculatedsuch that pixels in each of the one or more regions in which each pixelof the processing target image is included have substantially the samegain, and based on the calculated gain, a processed image is generatedby performing dynamic range compression on the processing target image.

In this way, according to the first image processing apparatus andmethod, substantially the same gain is calculated for pixels in each ofthe one or more regions of the processing target image in which eachpixel of the processing target image is included. Consequently, dynamicrange compression of the processing target image does not causeovershooting and undershooting arising from a large pixel valuevariation adjacent to the boundary of the one or more regions in whicheach pixel of the image is included. Therefore, a high quality processedimage without overshooting and undershooting may be obtained.

Here, the low frequency image represents a brightness variation of theprocessing target image in a manner such that the lower the frequencyband of the low frequency image, the more global will be the variation.Therefore, in order to calculate the gain for changing an overalldynamic range of the processing target image, it is preferable that thefrequency band of the low frequency image is as low as possible. On theother hand, if the low frequency image has a very low frequency band incomparison with that of the processing target image, the frequencydifference between the two images becomes large, and the blurred range,i.e., variation range of pixel levels in the low frequency image becomesvery large at the boundary between the regions in which each pixel ofthe processing target image is included, so that the use of the regionsetting result of the processing target image does not yield appropriategains.

According to the second image processing apparatus and method of thepresent invention, a plurality of low frequency images of differentfrequency bands is generated from a processing target image, and one ormore regions are set on at least one of those of the plurality of lowfrequency images up to a predetermined frequency band, which is higherthan a first frequency band, in which each pixel of the at least one lowfrequency image is included according to a level of each pixel of the atleast one low frequency image. Then, a first gain is calculated suchthat the lower the level of each pixel of a first low frequency image ofthe first frequency band the greater the gain and, based on a regionsetting result, a representative value of the first gains of pixels ineach of the one or more regions in which each pixel of a second lowfrequency image of a second frequency band, which is next higher thanthe first frequency band, is included is calculated as a second gain ofeach pixel of the second low frequency image.

Here, the first and second low frequency images are close to each otherin frequency band, so that the difference in blur degree is small.Therefore, the pixel level variation range in the first low frequencyimage becomes small at the boundary between the regions in which eachpixel of the second low frequency image is included, so that the gainsfor the second low frequency image may be calculated appropriately.

Further, in the second image processing apparatus and method of thepresent invention, the calculation of a new second gain is repeatedusing a low frequency image of the next higher frequency band than thatof the second low frequency image as a new second low frequency imageand the second gain as a new first gain to the predetermined frequencyband, and a second gain calculated by this is outputted as a final gainof each pixel of the processing target image.

Consequently, even where the frequency band of the low frequency imageis low, by gradually increasing the gain to be calculated and frequencyband for region splitting, the gain according to a global brightnessvariation of the processing target image may be calculated without beinginfluenced by the variation range of pixel levels at the boundarybetween the regions in which each pixel of the processing target imageis included. Accordingly, a high quality processed image withoutovershooting and undershooting may be obtained.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a schematic block diagram of a digital camera to which theimage processing apparatus according to a first embodiment of thepresent invention is applied, schematically illustrating theconfiguration thereof.

FIG. 2 illustrates a configuration of the imaging unit shown in FIG. 1.

FIG. 3 illustrates an example of a color filter.

FIG. 4 is a schematic block diagram of the gain processing unit shown inFIG. 1, schematically illustrating the configuration thereof.

FIG. 5 is a two-dimensional representation of the luminance values of animage for explaining gain calculation.

FIG. 6 is a two-dimensional representation of the luminance values of ablurred image for explaining gain calculation.

FIG. 7 illustrates a luminance value curve of an edge portion forexplaining gain calculation.

FIG. 8 illustrates a provisional gain curve.

FIG. 9 illustrates the state in which overshooting and undershootingoccur.

FIG. 10 is a two-dimensional representation of the provisional gainscalculated based on the blurred image.

FIG. 11 is a two-dimensional representation of the final gainscalculated based on the region setting result.

FIG. 12 illustrates provisional and final gain curves.

FIG. 13 illustrates a gain processing result according to the firstembodiment.

FIG. 14 is a flowchart illustrating processing performed in the firstembodiment.

FIG. 15 schematically illustrates processing performed in a secondembodiment.

FIG. 16 is a flowchart illustrating the processing performed in thesecond embodiment.

FIG. 17 illustrates region setting of a blurred image.

DESCRIPTION OF THE PREFERRED EMBODIMENTS

Hereinafter, embodiments of the present invention will be described withreference to the accompanying drawings. FIG. 1 is a schematic blockdiagram of a digital camera to which the image processing apparatusaccording to a first embodiment of the present invention is applied,illustrating the configuration thereof. As illustrated in FIG. 1,digital camera 1 according to the present embodiment includes imagingunit 2, imaging control unit 3, signal processing unit 4,compression/expansion unit 5, frame memory 6, medium control unit 7,internal memory 8, and display control unit 9.

FIG. 2 illustrates a configuration of imaging unit 2. As illustrated inFIG. 2, imaging unit 2 includes lens 20, aperture 21, shutter 22, CCD23, analog front-end (AFE) 24, and A/D converter 25.

Lens 20 includes a plurality of functional lenses, such as a focus lensfor bringing a subject into focus, a zoom lens for realizing a zoomfunction and the like, and positions of the lenses are controlled by anot shown lens drive unit.

The aperture diameter of aperture 21 is controlled by a not shownaperture drive unit based on the aperture data obtained by AEprocessing.

Shutter 22 is a mechanical shutter and driven by a not shown shutterdrive unit according to the shutter speed obtained by AE processing.

CCD 23 includes a photoelectric surface having multitudes of lightreceiving elements disposed two-dimensionally, and a light imagerepresenting a subject is formed on the photoelectric surface andsubjected to photoelectric conversion, whereby an analog image signal isobtained. A color filter having R, G, and B filters disposed regularlyis provided in front of CCD 23.

AFE performs noise reduction and gain adjustment (hereinafter, analogprocessing) on an analog image signal outputted from CCD 23.

A/D conversion unit 25 converts the analog image signal analog-processedby AFE to a digital signal. The image data obtained by converting theanalog image signal obtained by CCD 23 of imaging units 2 are RAW datain which each pixel has a density value of R, G, or B.

Imaging control unit 3 controls imaging after the release button isdepressed. It also performs AF and AE processing to set the focalposition, aperture value data and shutter speed when the release buttonis depressed half-way. When the release button is not depressed, imagingcontrol unit 3 controls imaging unit 2 to take a through image.

Signal processing unit 4 performs signal processing, such as whitebalance correction, tone correction, sharpness correction, and colorcorrection on the digital image data obtained by imaging unit 2.

Compression/expansion unit 5 generates an image file by compressing theimage data processed by signal processing unit 4, for example, in JPEGcompression format. A header that includes auxiliary information, suchas date and time of photographing, and the like, is attached to theimage file based on Exif format or the like.

Frame memory 6 is a work memory used when various types of processing,including the processing of signal processing unit 4, are performed onthe image data representing the image obtained by imaging units 2.

Medium control unit 7 gains access to recording medium 10 and controlsread/write operations of image file.

Internal memory 8 has stored therein various constants to be set indigital camera 1, programs to be executed by CPU 13, and the like.

Display control unit 9 causes image data stored in frame memory 6 or animage recorded in recording medium 10 to be displayed on monitor 11.

Digital camera 1 further includes gain processing unit 12. FIG. 4 is aschematic block diagram of gain processing unit 12, illustrating theconfiguration thereof. As illustrated in FIG. 4, gain processing unit 12includes level information calculation unit 31, low frequency imagegeneration unit 32, region setting unit 33, gain calculation unit 34,and multiplication unit 35.

Level information calculation unit 31 calculates the level of each pixelof processing target image S0 (hereinafter, also simply referred to as“image S0”) obtained by photographing and not yet subjected to thesignal processing. More specifically, level information calculation unit31 calculates luminance value Y of each pixel as the level by Formula(1) below.

Y=0.299R+0.587G+0.114B   (1)

Here, each pixel of image S0 has only one of R G B values, so that levelinformation calculation unit 31 calculates luminance value Y of eachpixel after calculating each of RGB values by performing interpolationoperations using RGB values of surrounding pixels. More specifically,for a pixel having a R value shown in FIG. 3, level informationcalculation unit 31 calculates G and B values by performinginterpolation operations using G values of four adjacent pixels in theleft/right and up/down directions and B values of four adjacent pixelsin the directions diagonal to the left/right and up/down directions. Fora pixel having a B value, it calculates G and R values by performinginterpolation operations using G values of four adjacent pixels in theleft/right and up/down directions and R values of four adjacent pixelsin the directions diagonal to the left/right and up/down directions. Fora pixel having a G value, it calculates R and B values by performinginterpolation operations using R values of two adjacent pixels in theleft/right or up/down direction and B values of two adjacent pixels inthe left/right or up/down direction.

Low frequency image generation unit 32 generates blurred image Sus0 byfiltering image S0 constituted by luminance values Y using a low-passfilter.

Region setting unit 33 sets one or more regions on image S0 in whicheach pixel of image S0 is included according to the magnitude ofluminance value Y of each pixel. More specifically, region setting unit33 calculates, with respect to each region setting target pixel of imageS0, an absolute value of difference between luminance value Y of thetarget pixel and luminance value Y of another pixel in a predeterminedrange surrounding the target pixel, and sets an region formed of pixelswhose absolute values of difference are within a predetermined thresholdvalue Th1 as the region in which the target pixel is included.

For example, as illustrated in FIG. 5, when image S0 has a diagonal edgeand each of luminance values Y in the upper right region is 255 and eachof luminance values Y in lower left region is 1, and if the shaded pixelin the center is the region setting target pixel, the region in whichthe target pixel is included is the upper right region formed of pixelshaving luminance value Y of 255, since the luminance value Y of 255 isthe same value as that of the target pixel. On the other hand, theluminance value Y of 1 differs largely from that of the target pixel, sothat the target pixel does not belong to the lower left region.

As shown in FIG. 6, in blurred image Sus0 of image S0 shown in FIG. 5,luminance values Y of the boundary area are reduced in the upper rightregion and increased in the lower left region, whereby the boundary isblurred.

Gain calculation unit 34 calculates provisional gains Gk based onblurred image Sus0 such that the darker the portion of image S0 havingsmaller luminance value Y, the greater the provisional gain. Forexample, with respect to an edge portion of image So shown in FIG. 7,the variation of luminance values Y becomes smooth in blurred imageSus0, as shown by a dashed line in FIG. 7, so that gain calculation unit34 calculates smoothly varying provisional gains Gk such that thegreater the luminance value of blurred image Sus0 the greater theprovisional gain, as shown in FIG. 8.

By multiplying the respective pixels of image S0 by provisional gains Gkcalculated in the manner as described above, the dynamic range, i.e.,contrast of image So may be reduced, so that a high contrast image of aperson taken against light can be turned into a high quality image bycorrecting halation in the background and dark collapsed person's face.

If the dynamic range compression is implemented by provisional gains Gkshown in FIG. 8, however, overshooting and undershooting occur adjacentto an edge included in image S0, as shown in FIG. 9, thereby imagequality is degraded.

Consequently, in the first embodiment, a representative value ofprovisional gains Gk for pixels in each of the one or more regions ofimage S0 in which each pixel of image S0 is included is calculated asfinal gain Gf based on a region setting result. For example, as shown inFIG. 10, where provisional gain Gk for each pixel in the boundary areain the upper right region is 0.3 and provisional gain Gk for each of theother pixels in the upper right region is 0.5, and provisional gain Gkfor each pixel in the boundary area in the lower left region is 5 andprovisional gain Gk for each of the other pixels in the lower leftregion is 10, the average value of provisional gains Gk of all pixels inthe upper right region is calculated as final gain Gf for the targetpixel in the center.

For each pixel in the upper right region, the average value ofprovisional gains Gk of all pixels is calculated as final gain Gf. Thiswill result in that gain Gf for each pixel in the upper right region is0.43 and gain Gf for each pixel in the lower left region is 5.8, asshown in FIG. 11.

Consequently, final gains Gf are calculated such that the gain variationin the boundary area becomes steeper than that of provisional gains Gk,as shown in FIG. 12. Accordingly, if dynamic range compression isperformed on image So using final gains Gf, overshooting orundershooting is prevented adjacent to an edge in image So, as shown inFIG. 13.

CPU 13 controls each unit of digital camera 1 according to a signal frominput/output unit 14 that includes an arrow key, various operationbuttons, and the release button.

Data bus 15 connects each unit of digital camera 1 and CPU 13, andvarious types of data and information in digital camera 1 are exchangedthrough the bus.

Processing performed in the first embodiment will now be described. FIG.14 is a flowchart illustrating the processing performed in the firstembodiment. Here, processing after an instruction to start photographingis issued by a full depression of the release button and digital imageS0 is generated will be described. When image S0 is obtained byphotographing, level information calculation unit 31 of gain processingunit 12 calculates the level of each pixel of image S0 (step ST1). Then,low frequency image generation unit 32 generates blurred image Sus0 ofimage S0 (step ST2), and region setting unit 33 sets one or more regionson image S0 in which each pixel of image S0 is included according to thelevel of each pixel (step ST3).

Then, gain calculation unit 34 calculates provisional gains Gk accordingto blurred image Sus0 (step ST4) and calculates final gains Gf based ona region setting result (step ST5). Multiplication unit 35 compressesthe dynamic range of image S0 by multiplying each pixel of image S0 bygain Gf, thereby generating dynamic range compressed image S1 (stepST6).

Next, signal processing unit 4 performs signal processing on dynamicrange compressed image S1, thereby generating processed image S2 (stepST7). Then, compression/expansion unit 5 generates an image file (stepST8) and medium control unit 7 records the image file on recordingmedium 10 (step ST9), thereby terminating the processing.

As described above, in the first embodiment, one or more regions are seton image S0 in which each pixel of image S0 is included according to thelevel of each pixel of image S0, whereby a substantially the same gainis calculated for each pixel in each of the one or more regions.Consequently, dynamic range compression of image S0 does not causeovershooting and undershooting arising from a large pixel valuevariation adjacent to the boundary of the one or more regions in whicheach pixel of image S0 is included. Therefore, high quality processedimage S2 without overshooting and undershooting may be obtained.

Next, a second embodiment of the present invention will be described.The configuration of a digital camera having an image processingapparatus according to the second embodiment is identical to that of thedigital camera having the image processing apparatus according to thefirst embodiment, so that it will not be elaborated upon further here.The second embodiment differs from the first embodiment only in theprocessing performed by gain processing unit 12.

The second embodiment differs from the first embodiment in that, in thefirst embodiment, only one blurred image is generated, while in thesecond embodiment, a plurality of blurred images of different frequencybands is generated.

FIG. 15 schematically illustrates processing performed in the secondembodiment. As shown in FIG. 15, low frequency image generation unit 32generates a plurality of blurred images Sus0 to Susn of differentfrequency bands from image S0 constituted by luminance values Y in thesecond embodiment. More specifically, low frequency image generationunit 32 generates a plurality of blurred images Sus0 to Susn byfiltering image S0 repeatedly using low-pass filters or by performing amultiple resolution conversion by wavelet transform or the like. Here,“n” represents the number of blurred images and the greater the “n” thehigher the frequency band of the blurred image.

Further, with respect to blurred images Sus1 to Susn other than blurredimage Sus0 of a lowest frequency band, region setting unit 33 sets oneor more regions on each of blurred images Sus1 to Susn in which eachpixel is included.

In the mean time, gain calculation unit 34 calculates provisional gainsG0 based on blurred image Sus0 of the lowest frequency band in the samemanner as in the first embodiment. Then, based on a region settingresult of blurred image Sus1 of the next higher frequency band than thatof blurred image Sus0, gain calculation unit 34 calculates arepresentative value of provisional gains G0 of pixels in each of theone or more regions of blurred image Sus1 as provisional gain G1 ofblurred image Sus1. The calculation of provisional gains G1 is performedin the same manner as the calculation of provisional gains Gk based onblurred image Sus0 and final gain Gf based on the region setting resultof image S0 in the first embodiment.

Then, based on the region setting result of blurred image Sus2 of thenext higher frequency band than that of blurred image Sus1, gaincalculation unit 34 calculates a representative value of provisionalgains G1 of pixels in each of the one or more regions of blurred imageSus2 as provisional gains G2 of blurred image Sus2.

Thereafter, gain calculation unit 34 repeats the calculation untilprovisional gains Gn for blurred image Susn of a highest frequency bandis calculated and outputs provisional gains Gn as final gains Gf.

Next, processing performed in the second embodiment will be described.FIG. 16 is a flowchart illustrating processing performed in the secondembodiment. Here, processing after an instruction to start photographingis issued by a full depression of the release button and digital imageS0 is generated will be described. When image S0 is obtained byphotographing, level information calculation unit 31 of gain processingunit 12 calculates the level of each pixel of image S0 (step ST11).Then, low frequency image generation unit 32 generates a plurality ofblurred images Sus0 to Susn from image S0 (step ST12), and regionsetting unit 33 sets one or more regions on each of images Sus1 to Susn,other than blurred image Sus0 of a lowest frequency band, in which eachpixel of each of blurred images Sus1 to Susn is included according tothe level of each pixel (step ST13).

Then, gain calculation unit 34 calculates provisional gains G0 based onblurred image Sus0 (step ST14), and sets the blurred image whose regionsetting result is used to first blurred image Sus1 (i=1, step ST15), andcalculates provisional gains Gi from provisional gains Gi-1 (initialvalue is G0) based on the region setting result of blurred image Susi(step ST16). Thereafter, a determination is made as to whether or notprovisional gains Gi are calculated based on the region setting resultof each of all blurred images, i.e., whether or not i=n (step ST17). Ifstep ST17 is negative, the blurred image whose region setting result isused is set to a blurred image of the next higher frequency band (i=i+1,step ST18), and the processing returns to step ST16.

If step ST17 is positive, multiplying unit 25 compresses the dynamicrange of image S0 by multiplying each pixel of image S0 by final gain Gf(Gn), thereby generating dynamic range compressed image S1 (step ST19).

Next, signal processing unit 4 performs signal processing on dynamicrange compressed image Sl, thereby generating processed image S2 (stepST20). Then, compression/expansion unit 5 generates an image file (stepST21) and medium control unit 7 records the image file on recordingmedium 10 (step ST22), thereby terminating the processing.

In the first embodiment, blurred image Sus0 represents a brightnessvariation of image S0 in a manner such that the lower the frequency bandof blurred image Sus0, the more global will be the variation. Therefore,in order to calculate the gain for changing an overall dynamic range ofimage S0, it is preferable that the frequency band of blurred image Sus0is as low as possible. On the other hand, if blurred image Sus0 has avery low frequency band in comparison with that of original image S0,the frequency difference between the two images becomes large, and theblurred range, i.e., level variation range in blurred image Sus0 becomesvery large at the boundary between the one or more regions in which eachpixel of image S0 is included, so that the use of the region settingresult of image S0 does not yield appropriate gains.

According to the second embodiment, a plurality of blurred images Sus0to Susn of different frequency bands is generated from image S0, andaccording to the level of each pixel of each of blurred images Sus1 toSusn having higher frequency bands than a lowest frequency band of theplurality of blurred images, one or more regions is set on each ofblurred images Sus1 to Susn in which each pixel is included. Then, fromprovisional gains G0 calculated based on blurred image Sus0 having alowest frequency band, provisional gains G1 is calculated based on theregion setting result of blurred image Sus1, and thereafter thecalculation is repeated until provisional gains Gn are calculated basedon the region setting result of blurred image Susn having a highestfrequency band.

Here, the frequency difference between blurred image Susi-1 and blurredimage Susi is small because their frequency bands are close to eachother. Therefore, the blurred range, i.e., variation range of pixellevels in blurred image Susi-1 becomes small at the boundary between theregions in which each pixel of image Susi is included, so that gains Gifor blurred image Susi may be calculated appropriately.

Therefore, by gradually increasing the gain to be calculated andfrequency band for region splitting, gain Gf according to a global levelvariation of image S0 may be calculated without being influenced by theblurred range, i.e., variation range of pixel levels in blurred imageSus0 at the boundary between the regions in which each pixel of image S0is included. Accordingly, high quality processed image S2 withoutovershooting and undershooting may be obtained.

In the first embodiment, final gain Gf is calculated from a provisionalgain calculated from blurred image Sus0 based on the region settingresult of image S0. Here, an arrangement maybe adopted in which anaverage value of each pixel of blurred image Sus0 in each of the one ormore regions in which each pixel of blurred image Sus0 is included iscalculated and a gain for each pixel is calculated based on the averagevalue such that the lower the level of each pixel the greater the gain.

That is, with respect to blurred image Sus0 calculated like that shownin FIG. 6, an average value of luminance values Y of blurred image Sus0is calculated for the upper right region in which the center pixel isincluded. FIG. 17 shows the average luminance value calculation resultof blurred image Sus0. Note that an average value of luminance values Yfor the lower left region is also calculated in FIG. 17. Then, withrespect to blurred image Sus0 for which average values are calculated,by calculating gains such that the darker the portion of image S0 havingsmaller luminance value Y the greater will be the pixel value, gainsthat do not cause overshooting or undershooting adjacent to an edge inimage S0 may be calculated, as in the first embodiment.

In the first and second embodiments, the image processing apparatus ofthe present invention is applied to a digital camera, but the imageprocessing apparatus may be used as a stand-alone apparatus. In thiscase, the image processing apparatus is provided with an interface, suchas a card reader, for receiving an image obtained by a digital camera orthe like.

Further, in the first and second embodiments, luminance value Y iscalculated as the pixel level, but RGB average value of each pixel, onlyG value, or maximum value of RGB values may be used as the pixel level.

Still further, in the first and second embodiments, an average value ofgains for pixels in each of one or more regions in which each pixel isincluded is calculated, but the weighted average according to thedistance from each pixel, intermediate value, or the like may be used,instead of the average value.

Further, in the first and second embodiments, a gain to be multiplied toeach pixel of image S0 is calculated, but a gain to be added to orsubtracted from each pixel may be calculated. In this case, gainprocessing unit 12 performs gain processing by adding the gain to orsubtracting from each pixel of image S0.

Still further, in the first and second embodiments, gains for theentirety of image S0 are calculated and dynamic rang compression isperformed for the entirety of image S0, but an arrangement may beadopted in which gains only for a high luminance region or low luminanceregion are calculated and the dynamic range of only the high luminanceregion or low luminance region is compressed.

Further, in the second embodiment, the final gain is calculated fromprovisional gain G0 with respect to blurred image Sus0 having a lowestfrequency band, but the frequency band for calculating the provisionalgain is not limited to the lowest frequency band and another frequencyband other than a predetermined highest frequency band may be used.Still further, it is not necessary to calculate the final gain usingregion setting results up to blurred image Susn of a highest frequencyband, and the final gain may be calculated using region setting resultsup to blurred image of a predetermined frequency band.

So far, apparatus 1 according to the embodiments of the presentinvention has been described, but a program for causing a computer tofunction as means corresponding to level information calculation unit31, low frequency image generation unit 32, region setting unit 33, gaincalculation unit 34, and multiplication unit 35, and to performprocessing like that shown in FIGS. 14, and 16 is another embodiment ofthe present invention. Further, a computer readable recording medium onwhich is recorded such a program is still another embodiment of thepresent invention.

1. An image processing apparatus, comprising: a region setting unit forsetting one or more regions on a processing target image in which eachpixel of the image is included according to a level of each pixel; a lowfrequency image generation unit for generating a low frequency image ofthe processing target image; a gain calculation unit for calculating again for each pixel of the processing target image such that the lowerthe level the greater the gain, wherein the unit calculates the gainsuch that pixels in each of the one or more regions in which each pixelof the processing target image is included have substantially the samegain based on a region setting result and a level of each pixel of thelow frequency image; and a processing unit for generating a processedimage by performing dynamic range compression on the processing targetimage based on the gain.
 2. The image processing apparatus of claim 1,wherein the gain calculation unit is a unit that calculates aprovisional gain for each pixel of the low frequency image such that thelower the level the greater the gain and, based on the region settingresult, calculates a representative value of the provisional gains ofpixels in each of the one or more regions in which each pixel of theprocessing target image is included as the gain for each pixel of theprocessing target image.
 3. The image processing apparatus of claim 1,wherein the gain calculation unit is a unit that calculates, based onthe region setting result, a representative value of pixels in each ofthe one or more regions in which each pixel of the low frequency imageis included, and calculates the gain for each pixel of the processingtarget image such that the lower the level of the representative valuethe greater the gain.
 4. An image processing apparatus, comprising: alow frequency image generation unit for generating a plurality of lowfrequency images of different frequency bands from a processing targetimage; a region setting unit for setting one or more regions on at leastone of those of the plurality of low frequency images up to apredetermined frequency band, which is higher than a first frequencyband, in which each pixel of the at least one low frequency image isincluded according to a level of each pixel of the at least one lowfrequency image; a gain calculation unit for calculating a first gainfor each pixel of a first low frequency image of the first frequencyband such that the lower the level the greater the gain, calculating,based on a region setting result, a representative value of the firstgains of pixels in each of the one or more regions in which each pixelof a second low frequency image of a second frequency band, which isnext higher than the first frequency band, is included as a second gainof each pixel of the second low frequency image, and outputting a secondgain calculated by repeating the calculation of a new second gain usinga low frequency image of the next higher frequency band than that of thesecond low frequency image as a new second low frequency image and thesecond gain as a new first gain to the predetermined frequency band as afinal gain of each pixel of the processing target image; and aprocessing unit for generating a processed image by performing dynamicrange compression on the processing target image based on the gain.
 5. Aphotographing apparatus, comprising: an imaging unit for obtaining aprocessing target image by photographing a subject; and the imageprocessing apparatus of claim
 1. 6. A photographing apparatus,comprising: an imaging unit for obtaining a processing target image byphotographing a subject; and the image processing apparatus of claim 4.7. An image processing method, comprising the steps of: setting one ormore regions on a processing target image in which each pixel of theimage is included according to a level of each pixel; generating a lowfrequency image of the processing target image; when calculating a gainfor each pixel of the processing target image such that the lower thelevel the greater the gain, calculating the gain such that pixels ineach of the one or more regions in which each pixel of the processingtarget image is included have substantially the same gain based on aregion setting result and a level of each pixel of the low frequencyimage; and generating a processed image by performing dynamic rangecompression on the processing target image based on the gain.
 8. Animage processing method, comprising the steps of: generating a pluralityof low frequency images of different frequency bands from a processingtarget image; setting one or more regions on at least one of those ofthe plurality of low frequency images up to a predetermined frequencyband, which is higher than a first frequency band, in which each pixelof the at least one low frequency image is included according to a levelof each pixel of the at least one low frequency image; calculating afirst gain for each pixel of a first low frequency image of the firstfrequency band such that the lower the level the greater the gain;calculating, based on a region setting result, a representative value ofthe first gains of pixels in each of the one or more regions in whicheach pixel of a second low frequency image of a second frequency band,which is next higher than the first frequency band, is included as asecond gain of each pixel of the second low frequency image; andoutputting a second gain calculated by repeating the calculation of anew second gain using a low frequency image of the next higher frequencyband than that of the second low frequency image as a new second lowfrequency image and the second gain as a new first gain to thepredetermined frequency band as a final gain of each pixel of theprocessing target image; and generating a processed image by performingdynamic range compression on the processing target image based on thegain.
 9. A computer readable recording medium on which is recorded aprogram for causing a computer to execute an image processing method,the method comprising the steps of: setting one or more regions on aprocessing target image in which each pixel of the image is includedaccording to a level of each pixel; generating a low frequency image ofthe processing target image; when calculating a gain for each pixel ofthe processing target image such that the lower the level the greaterthe gain, calculating the gain such that pixels in each of the one ormore regions in which each pixel of the processing target image isincluded have substantially the same gain based on a region settingresult and a level of each pixel of the low frequency image; andgenerating a processed image by performing dynamic range compression onthe processing target image based on the gain.
 10. A computer readablerecording medium on which is recorded a program for causing a computerto execute an image processing method, the method comprising the stepsof: generating a plurality of low frequency images of differentfrequency bands from a processing target image; setting one or moreregions on at least one of those of the plurality of low frequencyimages up to a predetermined frequency band, which is higher than afirst frequency band, in which each pixel of the at least one lowfrequency image is included according to a level of each pixel of the atleast one low frequency image; calculating a first gain for each pixelof a first low frequency image of the first frequency band such that thelower the level the greater the gain; calculating, based on a regionsetting result, a representative value of the first gains of pixels ineach of the one or more regions in which each pixel of a second lowfrequency image of a second frequency band, which is next higher thanthe first frequency band, is included as a second gain of each pixel ofthe second low frequency image; and outputting a second gain calculatedby repeating the calculation of a new second gain using a low frequencyimage of the next higher frequency band than that of the second lowfrequency image as a new second low frequency image and the second gainas a new first gain to the predetermined frequency band as a final gainof each pixel of the processing target image; and generating a processedimage by performing dynamic range compression on the processing targetimage based on the gain.